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EXECUTIVE  SUMMARY 


PROBLEM 

The  advent  of  multimedia  and  advancement  of  technology  provide  new  and  untested  tools 
for  improving  training  and  training  systems.  Specifically,  technologies  now  allow  for  easy 
application  of  visual  augmentation  to  computer  displays  that  may  enhance  performance  and 
learning.  However,  not  enough  is  known  about  the  implementation  of  display  augmentation 
techniques  which  lead  to  enhanced  team  performance. 

OBJECTIVE 

The  objective  of  the  current  experiment  was  to  determine  if  the  use  of  display 
augmentation,  specifically  highlighting  important  display  characteristics,  during  a  training 
intervention  could  improve  performance  on  a  team  task.  It  was  hypothesized  that  participants  in 
the  highlighted  condition  would  perform  better  than  those  in  the  other  two  conditions. 

APPROACH 

A  total  of  32  individuals  were  randomly  assigned  to  sixteen  2-person  teams.  Teams  were 
trained  to  perform  AIRTANDEM,  a  radar  simulation  task,  prior  to  watching  a  multimedia 
training  intervention  which  trained  teams  on  the  selection  of  targets  for  engagement.  The 
multimedia  training  intervention  included  video  vignettes  of  experts  performing  the  radar  task, 
still  firames  of  the  radar  screen,  and  a  transcript  of  the  experts’  communication.  After  the  final 
testing  trials,  participants  completed  a  questionnaire  that  was  designed  to  assess  the  degree  to 
which  the  highlighting  manipulation  was  successful. 

RESULTS 

Highlighting  the  transcript  reduced  the  average  amount  of  time  targets  were  in  the 
protected  area  significantly  when  compared  to  the  non-highlighted  transcript  condition  and  the 
control  condition.  Teams  which  received  highlighting  were  significantly  better  at  selecting 
targets  before  they  entered  protected  areas  and  were  faster  at  acting  on  targets  which  had  entered 
protected  areas.  On  average,  participants  rated  the  training  intervention  as  moderately  helpful. 

CONCLUSIONS 

The  present  experiment  demonstrated  that  highlighting  textual  information  during  a 
training  intervention  can  positively  impact  performance.  These  results  indicate  that  the  positive 
benefits  in  performance  were  due  to  the  display  augmentation  rather  than  the  presence  of  textual 
information  on  the  display  or  to  the  experts’  communication.  These  results  suggest  that 
highlighting  strategies  within  text  may  help  trainees  acquire  strategies  more  quickly,  focus 
attention  on  critical  aspects  of  the  task,  and  provide  a  framework  for  learning. 
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RECOMMENDATIONS 

Additional  research  is  needed  to  investigate  alternative  forms  of  display  augmentation  to 
enhance  the  selection  of  material  for  schema  encoding.  Additional  research  is  needed  to 
determine  the  long-term  effects  of  highlighting  display  elements. 


i 
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INTRODUCTION 


PROBLEM 

The  advent  of  multimedia  and  advancement  of  technology  provide  new  and  untested  tools 
for  improving  training  and  training  systems.  Specifically,  technologies  now  allow  for  easy 
application  of  visual  augmentation  to  computer  displays  that  may  enhance  learning  and 
performance.  However,  not  enough  is  known  about  the  implementation  of  display  augmentation 
techniques  that  lead  to  enhanced  team  performance.  Trainers  and  instructional  developers  need 
to  take  advantage  of  current  technology  and  employ  it  within  the  proper  settings  in  order  to 
optimize  its  utility  and  provide  the  most  benefit  for  trainees.  It  has  become  common  practice  to 
use  these  technologies  haphazardly.  In  order  to  achieve  optimal  learning  from  display 
augmentation  techniques,  learning  theory  must  guide  the  implementation  of  display 
augmentation. 

OBJECTIVE 

The  current  investigation  proceeds  from  a  critical  review  of  cognitive  theory  to  drive  the 
use  of  display  augmentation  in  training.  This  experiment  was  designed  and  executed  in  order  to 
determine  the  effectiveness  of  using  a  display  augmentation  technique,  specifically  highlighting, 
empirically  during  a  multimedia  training  intervention  on  team  performance. 

BACKGROUND 

Schemas  and  Display  Augmentation 

Schema  theory  provides  a  powerful  organizing  framework  for  enhancing  the  efficacy  of 
visual  technology  use  in  the  provision  of  training.  According  to  schema  theory,  the  application 
of  different  visual  technologies  may  vary  depending  on  the  skill  that  is  being  trained.  A  schema 
refers  to  the  “general  knowledge  a  person  possesses  about  a  particular  domain”  and  “allows  for 
the  encoding,  storage,  and  retrieval  of  information  related  to  that  domain”  (Alba  &  Hasher,  1983, 
p.  203). 


Schemas  are  useful  ways  to  organize  and  reason  about  large  complex  knowledge  bases. 

In  order  for  a  schema  to  guide  processing  it  must  be  active  in  memory.  Once  active,  it  guides 
subsequent  information  processing  via  four  encoding  processes;  selection,  abstraction, 
interpretation,  and  integration  (Alba  &  Hasher,  1983).  Information  from  the  environment  is 
selected  for  further  processing  depending  on  its  relevance  to  the  activated  schema.  Following  the 
selection  process,  meaning  is  derived  from  the  relevant  stimuli  during  an  abstraction  process, 
which  produces  a  representation  of  only  the  crucial  aspects  of  the  stimulus.  Meaning  is  then 
interpreted  in  the  context  of  pre-existing  knowledge  (i.e.,  the  active  schema).  This  contextually 
dependent  interpretive  process  allows  the  derivation  of  inferences  about  the  current  environment. 
Finally,  the  integration  process  refers  more  directly  to  learning  processes  and  how  memory  traces 
are  modified  through  experience.  Integration  occurs  when  a  new  schema  is  formed  or  an  existing 
one  is  modified.  This  information  is  stored  and  now  available  for  subsequent  use. 
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Visual  technologies  when  used  appropriately  can  have  a  powerful  impact  on  these 
encoding  processes.  It  has  been  suggested  that  less  experienced  people  have  greater  difficulty 
than  more  experienced  people  in  selecting  task-relevant  cues  that  lead  to  the  activation  of 
appropriate  schemata  (Bransford  &  Nitsch,  1978,  as  cited  by  Alba  and  Hasher,  1983)  while 
experts  store  information  with  cues  that  can  later  expedite  retrieval  (Ericsson  &  Staszewski, 

1989).  Schema  theory  indicates  the  need  for  displays  that  distinguish  important  cues  from 

irrelevant  ones  within  a  complex,  stimulus  array  during  training.  This  directly  suggests  an 

application  of  visual  technologies  to  alleviate  the  novice’s  performance  deficit.  For  example,  • 

visual  technologies  that  highlight  important  information  may  increase  cue  saliency,  which  in  turn 

assists  in  the  selection  of  appropriate  schemata  to  guide  subsequent  information  processing.  i 

As  a  first  step,  training  interventions  should  target  the  activation  of  task-appropriate 
schemata.  Research  indicates  this  can  be  accomplished,  for  example,  by  using  training 
interventions  such  as  advanced  organizers  or  advice  which  frame  an  upcoming  experience  by 
relating  new  information  to  prior  knowledge,  thus  activating  relevant  schemata  (Kraiger,  Salas, 

&  Caimon-Bowers,  1995;  Phye,  1989;  Phye  &  Sanders,  1994).  After  task-relevant  schemata  are 
active  in  memory,  novices  can  be  assisted  by  technologies  that  distinguish  between  important 
and  irrelevant  environmental  cues.  For  example,  visual  technologies  that  highlight  important 
information  may  increase  cue  saliency,  which  in  turn  assists  in  the  selection  of  appropriate 
information  for  subsequent  processing. 

It  can  be  argued  that  influencing  the  selection  encoding  process  would  have  the  most 
effect  on  learning  because  if  information  is  not  selected  for  encoding,  then  it  will  be  unavailable 
for  further  processing.  Consequently,  an  understanding  of  display  augmentation  that  positively 
impacts  the  selection  process  is  warranted.  There  is  support  that  display  augmentation  can 
benefit  the  acquisition  of  flight  skills  (Lintem  &  Koonce,  1992;  Lintem,  Roscoe,  Koonce,  & 

Segal,  1990;  Lintem,  Roscoe,  &  Sivier,  1990;  Taylor,  Lintem,  Koonce,  Kaiser,  &  Morrison, 

1991)  and  influence  the  internal  representation  (schema)  developed  during  a  second  order 
tracking  task  (Eberts,  1983;  1988;  Eberts  &  Schneider,  1985)  and  visual  search  tasks  (Fisher  & 

Tan,  1989;  Fisher,  Coury,  Tengs,  &  Duffy,  1989)  in  individuals. 

Team  Schemas  and  Display  Augmentation 

Given  the  trend  toward  the  use  of  work  teams  to  accomplish  organizational  goals  (Salas,  • 

Dickinson,  Converse,  &  Tannenbaum,  1992),  it  is  becoming  increasingly  important  to 
understand  the  defining  characteristics  of  teams  in  order  to  optimize  team  training.  Despite  the 
increasing  use  of  teams  and  a  resurgence  of  team  training  research,  there  remains  a  lack  of 
behavioral  guidelines  to  guide  team  training  design  (Swezey  &  Salas,  1992).  With  the 
established  need  to  understand  teams,  researchers  have  begun  to  address  issues  of  team  cognition 
(Hinsz,  Tindale,  &  Vollrath,  1997).  Concepts,  such  as  team  schemas  (Rentsch  &  Hall,  1994)  and 
shared  mental  models  (Cannon-Bowers,  Salas,  &  Converse,  1993)  have  been  introduced  as 
guiding  forces  in  team  training  research. 
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As  an  extension  of  the  schema  concept  discussed  previously,  Rentsch  and  Hall  (1994) 
define  team  schema  as  the  “common  understandings  among  team  members  that  occur  when  there 
is  substantial  overlap  or  complementarity  in  the  content  and  organization  of  their  team-related 
knowledge”  (p.  232).  The  assumption  that  team  members  with  similar  schemas  share  the  same 
teamwork  knowledge  with  the  same  organization  perform  better  as  a  team  is  inherent  in  this 
definition  (Rentsch  &  Hall,  1994).  To  date,  there  is  limited  empirical  support  demonstrating  a 
relationship  between  team  member  schemas  and  team  effectiveness. 

Rentsch  and  Hall  (1994)  have  identified  a  teamwork  schema  that  appears  relevant  to 
teams.  Rentsch,  Heffner,  and  Duffy  (1994)  refer  to  teamwork  schema  as  knowledge  about 
elements  of  teamwork,  such  as  interdependence,  cooperation,  and  communication.  Each 
individual’s  abstraction  of  the  team  schema  could  vary  depending  upon  the  level  of  experience 
the  individual  has  working  in  teams  or  the  type  of  team  to  which  the  individual  belongs.  In  fact, 
Rentsch,  Heffner,  and  Duffy  (1994)  found  that  high  experience  team  members  represented  their 
teamwork  schema  using  more  abstract  defining  dimensions  than  low  experienced  team  members. 
According  to  Rentsch  (1993),  similar  teamwork  schema  predicted  three  aspects  of  team 
effectiveness:  client  satisfaction,  member  growth,  and  team  viability. 

Rentsch  and  Hall  (1994)  hypothesized  that  agreement  in  team  member  schemas  would 
enhance  team  performance  because  the  similar  schemas  would  decrease  the  amount  of  time 
members  spent  in  deliberation  over  team  issues  and  performance  standards.  Recently,  Jenkins 
and  Rentsch  (1995)  also  identified  accuracy  as  an  important  component  of  team  member 
schemas.  These  authors  speculate  that  accurate  schemas  will  enable  members  to  anticipate  and 
understand  the  actions  of  team  members. 

According  to  the  Team  Evolution  and  Maturation  Model,  teams  develop  two  kinds  of 
skills:  teamwork  and  taskwork  skills  (Morgan,  Glickman,  Woodard,  Blaiwes,  &  Salas,  1986). 
Rentsch  and  her  colleagues  have  confined  their  research  to  teamwork  knowledge.  Taskwork 
knowledge,  however,  is  equally  important  in  designing  team  training  that  will  optimize  team 
performance.  Taskwork  knowledge  includes  those  task-related  skills  that  team  members  must 
understand  and  acquire  for  task  performance.  There  is  growing  evidence  that  team  members 
must  have  a  shared  understanding  of  task  as  well  as  team  demands  in  order  to  coordinate  their 
efforts,  facilitate  commimication,  and  consequently,  perform  effectively. 

The  concept  of  a  team  schema  is  congruent  to  that  of  an  individual  schema.  Therefore,  it 
is  appropriate  to  infer  that  the  four  schema  encoding  processes  that  were  described  previously 
would  be  in  operation  for  team  schemas.  In  order  to  develop  team  schemas,  whether  they  are 
teamwork  or  taskwork  related,  information  must  be  selected  for  inclusion,  abstracted  and 
interpreted  for  meaning,  and  integrated  into  existing  schemata.  These  encoding  processes  should 
also  be  subject  to  factors  that  may  either  facilitate  or  hinder  schema  development  and  learning. 

As  described  above,  it  has  been  suggested  that  display  augmentation  should  aid  in  the 
selection  of  information  for  inclusion  in  one’s  schemata.  In  the  case  of  teams,  display 
augmentation  should  facilitate  the  selection  of  information  that  should  be  shared  by  the  team 
members  and  incorporated  into  the  team’s  schema  for  the  task  at  hand.  The  display 
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augmentation  should  make  the  task  demands  that  should  be  shared  by  the  team  members  more 
salient;  thus  contributing  to  enhanced  communication  and  coordination  between  team  members. 

The  Current  Experiment 

Training  as  it  is  performed  in  many  settings  is  often  observational  and  text-based.  Given 
the  trend  toward  computer  based  delivery  of  training  programs  (Bassi,  Benson,  &  Cheney,  1996), 
there  will  be  an  increased  use  of  computer  presented  text-based  training  programs.  It  is  likely 
that  many  training  programs  will  be  multimedia  presentations.  Since  many  multimedia  • 

development  programs  easily  allow  for  display  augmentation,  it  is  essential  to  determine  if 
beneficial  effects  can  be  found  in  text-based  training  settings.  One  display  augmentation  • 

approach  beginning  to  receive  attention  fi'om  scientists  in  this  area  is  highlighting  display 
elements.  Highlighting  refers  to  the  practice  of  contrasting  a  display  element  (i.e.,  text)  with  the 
display  background  (Coiy,  1 990).  It  has  been  suggested  that  highlighting  “aids  in  cue 
differentiation  by  calling  attention  to  important  words  and  phrases”  without  adding  additional 
visual  complexity  to  computer  displays  (Cory,  1990,  p.  25).  As  argued  above,  highlighting 
should  facilitate  the  selection  of  information  for  inclusion  into  a  trainee’s  as  well  as  the  team’s 
taskwork  schema  leading  to  more  rapid  and  accurate  incorporation  into  performance  (i.e., 
learning).  However,  there  is  limited  empirical  data  to  support  the  notion  that  highlighting 
enhances  performance  in  individuals  or  teams  (Fisher  &  Tan,  1989;  Fisher,  Coury,  Tengs,  & 

Duffy,  1989;  Hessler,  1972).  Fisher  and  Tan  (1989)  have  determined  empirically  that 
highlighting  attracts  attention  initially  to  highlighted  elements  on  a  display.  These  authors  used  a 
visual  search  task  in  which  participants  needed  only  to  identify  a  target  digit  from  among  a  series 
of  five  digits.  The  current  experiment,  however,  seeks  to  determine  whether  highlighting  the 
target  cues  (within  textual  information)  to  which  trainees  should  attend  influences  their 
performance.  Consequently,  this  experiment  is  investigating  the  beneficial  effects  of 
highlighting  on  team  performance. 

ORGANIZATION  OF  THE  REPORT 

This  technical  report  documents  a  research  project  investigating  the  use  of  highlighting 
display  elements  during  training  to  enhance  learning  and  performance.  First,  the  methodology  of 
the  empirical  research  project,  including  a  description  of  the  participants  and  materials,  is 
described.  Then,  the  statistical  analysis  of  the  data  is  reported.  The  results  are  followed  by  the 
conclusions  and  recommendations  that  could  be  deduced  from  the  data.  A  participant  • 

demographics  form,  the  training  materials,  and  an  assessment  questionnaire  which  were 
developed  as  part  of  this  research  project  are  included  in  appendices  A,  B,  C,  and  D. 
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METHOD 


PARTICIPANTS 

Participants  were  recruited  from  an  introductory  psychology  course  at  the  University  of 
Central  Florida  and  were  offered  course  extra  credit  for  their  participation.  A  total  of  32 
individuals  (22  females  and  10  males)  were  randomly  assigned  to  two-person  teams.  There  were 
a  total  of  16  teams  who  completed  the  experiment.  Participants  described  themselves  as  either 
juniors  or  seniors  in  college  and  ranged  in  age  from  20  to  48  years  of  age.  The  mean  age  was  25 
years  old.  GPA  ranged  from  2.0  to  3.99,  with  a  mean  of  3.0.  Twenty-one  of  the  participants 
were  majoring  in  psychology;  the  remaining  eleven  participants  were  majoring  in  subjects  other 
than  psychology.  Participants  ranged  in  computer  experience  from  having  used  computers  a  few 
times  to  using  computers  everyday.  Participants  also  ranged  in  their  experience  v^dth  video 
games,  some  reported  never  playing  video  games  while  others  reported  playing  video  games 
several  times  a  week. 

MATERIALS 

Simulation  Description 

Two  networked  IBM  compatible  computers  were  used  in  conjunction  with 
AIRTANDEM,  a  program  adapted  from  TANDEM  (Weaver,  Morgan,  &  Hall,  1993). 
AIRTANDEM  is  a  low-fidelity  simulated  radar  task  that  mimics  the  operations  of  the  E-2C.  In 
this  simulation,  targets  appear  as  ‘blips’  on  a  radar  screen.  Either  team  member  may  select 
targets  by  pressing  the  right  button  on  a  computer  mouse.  Once  a  target  is  selected,  the  team 
members  must  determine  the  type  of  craft  (air,  sea,  or  land),  its  threat  level  (civilian,  military,  or 
unknown)  and  its  intent  (peaceful,  hostile,  or  fleet  member).  These  determinations  are  made  by 
gathering  information  available  in  pull-down  menus  located  in  the  upper  right  hand  comer  of  the 
eomputer  sereen. 

Forms  and  Questionnaires 

Additional  materials  used  in  this  investigation  include:  1)  a  demographics  form  that  is 
used  to  obtain  personal  information  about  each  participant;  2)  video-based  training  on  the 
operation  of  AIRTANDEM;  3)  a  multimedia  training  intervention  was  developed  using  video 
vignettes  and  Microsoft  Power  Point;  and  4)  a  questionnaire  that  was  designed  to  assess  the 
degree  to  which  the  highlighting  manipulation  was  successful.  The  demographics  form,  training 
materials,  and  the  questiormaire  are  included  in  Appendices  A,  B,  C,  and  D  respeetively. 

Video  Vignettes 

Two  experts  were  videotaped  as  they  interacted  with  the  AIRTANDEM  simulation  task. 
Selected  clips,  which  described  the  proeess  the  team  of  experts  was  using  to  select  targets  for 
identification  and  prosecution,  were  digitized  and  imported  into  a  Microsoft  Power  Point 
presentation  in  order  to  produce  the  video  vignettes.  The  video  showed  two  experts  sitting  at 
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computer  facing  one  another.  Their  communication  was  also  recorded  and  digitized  along  with 
the  video.  Participants  could  see  and  hear  the  experts  as  they  performed  the  task.  The  vignettes 
also  included  still  screens  from  the  AIRTANDEM  radar  scope  which  were  attained  using  the  full 
screen  capture  feature  available  on  Paint  Shop  Pro.  The  target  that  the  experts  were  selecting 
was  highlighted.  To  complement  the  video  vignettes,  the  transcript  of  the  experts’ 
communication  was  added  into  the  presentation  and  appropriate  text  was  highlighted  in  a  high 
contrast  color  for  the  experimental  condition. 

PROCEDURE  * 

Sixteen  two-person  teams  were  trained  to  perform  AIRTANDEM,  a  multi-player  radar  • 

simulation  task  (Weaver  et  al.,  1993).  After  completing  four  practice  trials,  teams  were 
randomly  assigned  to  one  of  three  training  conditions  (highlighted  transcript,  non-highlighted 
transcript,  or  no  transcript  control).  Teams  in  all  conditions  were  shown  a  multimedia  training 
intervention  that  was  designed  to  aid  teams  in  the  selection  of  targets  for  engagement, 
specifically  teams  were  instructed  to  attend  to  the  location  and  characteristics  of  targets.  The 
multimedia  presentation  contained  video  vignettes  of  experts  performing  the  task  and  video 
snapshots  of  the  radar  simulation  screen.  The  snapshots  of  the  radar  simulation  provided  a  visual 
representation  of  the  target  the  experts  were  selecting  as  well  as  the  other  targets  in  the 
immediate  vicinity.  The  target  that  was  being  discussed  was  highlighted  for  participants  in  each 
condition.  In  addition  to  the  screen  snapshots  and  the  vignettes,  which  included  video  as  well  as 
audio  of  their  communication,  two  groups  viewed  a  transcript  of  the  experts’  communication  that 
was  displayed  while  the  vignette  played.  The  transcript  allowed  participants  to  read  along  as 
they  watched  and  heard  the  experts  perform  the  task.  For  one  group,  the  important  phrases  were 
highlighted  in  the  transcript  (highlighted  transcript).  Participants  were  simply  told  to  watch  the 
presentation.  The  experimenter  did  not  point  out  or  mention  the  highlighted  text.  Similarly, 
participants  were  not  told  that  the  important  cues  to  which  they  should  attend  were  highlighted  in 
the  transcript.  The  second  group  viewed  the  transcript  that  was  not  highlighted  (non-highlighted 
transcript).  The  third  group  did  not  view  a  transcript  of  the  experts’  communication;  they  only 
heard  the  experts’  conversation  and  viewed  the  screen  snapshots  and  video  vignettes  (control). 

In  the  highlighted  transcript  condition,  critical  information  about  the  characteristics  of  the  targets 

that  should  be  selected  for  engagement  was  highlighted.  The  authors  determined  the  “critical” 

information  that  was  highlighted  in  each  vignette  based  on  their  judgment  as  to  the  importance  of 

the  meaning  conveyed  by  the  phrase.  Phrases  that  described  the  target’s  characteristics  and/or  an 

action  that  should  be  taken  were  highlighted.  For  example,  the  experts  selected  targets  that  were  • 

described  as  “heading  straight  toward  us”.  The  training  intervention  lasted  approximately  twenty 

minutes.  At  the  beginning  of  the  presentation,  the  experimenter  explained  each  piece  of  the 

information  that  was  displayed  on  the  screen.  Then,  the  experimenter  controlled  the  presentation 

of  each  of  the  video  vignettes  by  starting  the  video  clip.  The  experimenter  interacted  with  the 

trainees  during  the  presentation  of  each  slide  by  answering  questions  and  providing  additional 

explanations  when  necessary.  After  the  presentation,  the  experimenter  answered  trainee’s 

questions.  After  the  intervention,  teams  performed  two  additional  trials.  Participants  then  filled 

out  a  questionnaire  concerning  the  effectiveness  of  the  training  intervention. 
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RESULTS 

MANIPULATION  CHECK 

A  manipulation  check  verified  that  all  teams  in  the  highlighted  transcript  condition 
detected  the  highlighting.  These  participants  rated  the  highlighting  as  moderately  helpful  in 
making  strategies  obvious  to  them  (x  =  2.8  out  of  5).  The  dependent  measures  for  performance 
were  (a)  the  total  amount  of  time  targets  spent  in  the  protected  area  around  the  ship  and  (b)  the 
average  time  that  each  target  spent  in  the  protected  area  before  a  final  action  was  taken  to  shoot 
4  the  target  or  clear  it  through  the  air  space.  These  measures  were  used  because  they  indicate  that 

targets  were  close  to  the  participant’s  own  ship. 

*  PERFORMANCE  DATA 

A  multivariate  analysis  of  covariance  (MANCOVA)  was  performed,  using  the  total 
amount  of  time  targets  spent  in  the  protected  area  and  the  average  time  that  each  target  spent  in  it 
during  the  post-training  trial  (i.e.,  trial  5)  as  the  dependent  variables.  Both  classes  of 
performance  data  during  the  pre-training  trial  (i.e.,  trial  4)  were  entered  as  covariates  to  account 
for  individual  differences  in  ability  between  the  teams.  Using  Wilks’  criterion,  the  regression  for 
the  combined  covariates  was  significant,  F(4, 20)  =  13.72,  p  <  .001 .  Univariate  regressions  for 
both  classes  of  performance  data  also  showed  significant  relationships  between  performance  in 
the  pretraining  trial  and  in  the  post-training  trial,  F(2,  1 1)  =  47.12,  p  <  .001  [the  total  amount  of 
time  targets  spent  in  the  protected  area],  and  F(2, 11)  =  55.98,  p  <  .001  [average  time  each  target 
spent  in  the  protected  area]. 

After  adjusting  for  the  covariates,  the  combined  adjusted  means  for  both  performance 
variables  during  the  post-training  trial  showed  the  patterns  indicated  in  Figure  1 .  The  patterns  of 
means  were  in  the  expected  direction,  with  teams  in  the  control  condition  allowing  targets  to 
spend  somewhat  longer  amounts  of  time  in  the  protected  areas  than  those  in  the  non-highlighted 
transcript  condition,  and  those  teams  in  the  highlighted  transcript  condition  allowing  the  targets 
to  spend  significantly  less  time  in  the  protected  area  than  teams  in  the  other  two  conditions. 

Using  Wilks’  criterion,  the  combined  dependent  variables  were,  in  fact,  significantly  affected  by 
the  training  condition,  F(4, 20)  =  2.70,  p  <  .05  (one-tailed).  Univariate  F-tests  also  indicated  that 
for  both  performance  measures  individually,  training  condition  had  a  significant  effect,  F(2,  11) 

=  4.65,  P  <  .02  (one-tailed)  [i.e.,  total  amount  of  time  targets  spent  in  the  protected  area]  and  F(2, 
«  11)  =  2.92,  p  <  .05  (one-tailed)  [i.e.,  the  average  time  each  target  spent  in  protected  area]. 


Insert  Figures  1  &  2  about  here 


Fisher  LSD  post-hoc  tests  indicated  that,  with  respect  to  the  total  amount  of  time  targets 
spent  in  the  protected  area,  highlighting  of  the  transcript  led  to  significantly  lower  total  amounts 
of  time  than  were  found  in  either  the  non-highlighted  transcript,  t(13)  =  2.78,  p  <  .01  (one-tailed), 
or  the  control  groups,  t(13)  =  2.65,  p  <  .05  (one-tailed).  The  difference  between  the  non- 
highlighted  transcript  and  the  control  groups,  however,  was  not  significant,  t(13)  =  0.23,  ns. 


13 


Technical  Report  97-004 


With  respect  to  the  average  time  each  target  spent  in  the  protected  area,  the  same  pattern 
of  results  was  found;  Highlighting  the  transcript  reduced  the  average  amount  of  time  targets 
were  in  the  protected  area  significantly  when  compared  to  the  non-highlighted  transcript 
condition,  t(13)  =  2.05,  p  <  .05  (one-tailed),  and  the  control  condition,  t(13)  =  2.20,  p  <  .05  (one- 
tailed).  The  difference  between  the  non-highlighted  transcript  and  the  control  conditions, 
however,  was  not  significant,  t(13)  =  0.48,  ns. 

QUESTIONNAIRE  DATA  ♦ 

Analysis  of  the  assessment  questionnaire  revealed  that,  on  average,  the  participants  § 

considered  the  highlighting  moderately  helpful  for  making  the  targeted  behaviors  more  obvious 
to  them  (mean  =  2.9).  They  judged  the  training  intervention  to  be  moderately  helpful  in  teaching 
them  which  targets  to  select  for  prosecution  (mean  =  2.7).  On  average,  participants  also  rated  the 
overall  training  intervention  as  moderately  helpful  (mean  =  2.7).  Similarly,  participants  stated 
that  they  would  moderately  recommend  using  this  technique  for  team  training  (mean  =  2.7). 

Despite  the  measured  increases  in  performance,  participants  perceived  the  training  intervention 
to  be  only  moderately  helpful  in  enhancing  their  performance. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 

The  present  experiment  employed  highlighting  as  a  technique  for  aiding  teams  in  the 
selection  of  targets  for  engagement  in  a  radar  simulation  task.  One  previous  experiment  failed  to 
find  significant  performance  improvements  when  using  a  highlighted  textual  display  (Cory, 

1990).  The  present  experiment,  however,  showed  that  highlighting  textual  information  during  a 
^  training  intervention  can  positively  impact  performance.  Teams  that  received  highlighting  were 

significmtly  better  at  selecting  targets  before  they  entered  protected  areas  and  were  faster  at 
^  acting  on  targets  that  had  entered  protected  areas.  Additionally,  these  results  indicate  that  the 

positive  benefits  in  performance  were  due  to  the  display  augmentation  rather  than  the  presence  of 
textual  information  on  the  display  or  to  the  experts’  communication.  These  results  suggest  that 
highlighting  strategies  within  text  may  help  trainees  acquire  strategies  more  quickly,  focus 
attention  on  critical  aspects  of  the  task,  and  provide  a  framework  for  learning. 

The  positive  benefits  that  were  found  from  highlighting  display  elements  adds  credence 
to  the  hypothesis  that  shared  taskwork  knowledge  can  benefit  team  performance.  As  evidenced 
by  the  faster  average  and  total  times  to  react  to  targets  in  the  protected  area,  it  can  be  inferred 
from  these  results  that  teams  which  received  the  augmented  displays  coordinated  better  than 
teams  in  the  non-highlighted  and  control  conditions.  This  also  implies  that  the  highlighted 
information  was  selected  for  inclusion  into  a  team  schema  for  taskwork  knowledge.  Before 
making  any  conclusive  statements  about  the  use  of  display  augmentation  to  aid  in  the  selection  of 
material  for  inclusion  into  team  taskwork  schemas,  a  more  rigorous  examination  of  the  team 
member’s  knowledge  structures  is  needed. 

Despite  the  beneficial  performance  effects  that  were  found,  team  members  perceived  the 
training  intervention  to  be  only  moderately  helpful  in  helping  them  to  acquire  this  task.  These 
results  raise  more  questions  concerning  the  use  of  display  augmentation  in  multimedia  training 
materials.  Trainee  reactions  to  the  training  are  important  in  training  design,  even  though  they 
may  not  be  an  indicant  of  what  or  how  much  the  trainee  is  learning.  In  examining  reasons  why 
trainees  formd  the  training  intervention  only  moderately  helpful,  the  design  of  each  POWER 
POINT  slide  was  considered.  For  instance,  the  issue  of  the  amount  of  information  that  the 
trainees  received  during  the  multimedia  presentation  may  have  overwhelmed  them.  The 
§  multimedia  presentation  as  developed  in  this  experiment  included  video  vignettes, 

AIRTANDEM  screen  shots,  and  text  (except  for  the  control  group).  Perhaps,  the  highlighting 
was  viewed  as  only  moderately  helpful  because  all  of  the  additional  material  that  was  included 
served  as  distractions,  although  not  detrimentally  so,  from  the  display  augmentation. 

In  conclusion,  these  results  suggest  that  highlighting  a  visual  display  can  be  beneficial  as 
a  training  tool.  Highlighting,  as  employed  in  this  experiment,  provided  an  inexpensive 
methodology  for  enhancing  training  and  performance  upon  which  trainers  may  be  able  capitalize. 
However,  additional  research  is  warranted  to  determine  the  circumstances  or  contexts  under 
which  highlighting  should  be  implemented;  the  cognitive  processes  that  are  impacted  by  the 
display  augmentation;  and  the  long-term  benefits  that  may  result. 


15 


Technical  Report  97-004 


RECOMMENDATIONS 

This  investigation  into  the  use  of  display  augmentation  to  enhance  learning  and 
performance  has  yielded  several  significant  findings  that  can  lead  to  recommendations  for  the  use 
of  highlighting  in  a  multimedia  training  intervention  and  also  for  additional  research.  First  of  all, 
highlighting  can  be  used  to  make  phrases  stand  out  from  other  textual  information.  Highlighting 
textual  information  in  this  manner  can  aid  in  the  selection  of  material  that  is  encoded  in  a 
trainee’s  schemata.  Since  on  average  participants  judged  the  training  intervention  as  moderately 
helpful,  it  may  be  important  to  note  that  despite  the  demonstrated  benefits  of  the  intervention 
trainee’s  may  have  gained  more  from  an  alternative  intervention  format. 

Additional  research  is  needed  to  investigate  alternative  forms  of  display  augmentation  to 
enhance  the  selection  of  material  for  schema  encoding.  Display  augmentation  and  more 
specifically,  highlighting,  are  often  used  in  computer  based  training  programs,  but  limited 
empirical  research  exists  which  sheds  light  on  the  most  optimal  techniques  that  can  be  used  for 
differing  tasks.  The  long-term  effects  of  this  type  of  training  intervention  must  also  be 
investigated.  Due  to  the  experimental  design  employed,  long-term  effects  were  not  investigated. 
The  beneficial  effects  obtained  in  the  current  experiment  may  be  subject  to  rapid  decay. 
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Transcript  highlighted 
Transcript 


Figure  1.  Adjusted  means  for  average  time  spent  in  protected  area. 


Transcript  highlighted 
Transcript 


Figure  2.  Adjusted  means  for  total  time  spent  in  protected  area. 
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APPENDIX  A 
DEMOGRAPHICS  FORM 


Identification  Number: 


Demographic  Data  Form  for  AirTandem 

1 .  Social  Security  Number:  _ 

2.  Gender:  Male  Female 

3.  Age:  _ 

4.  Major:  _ 

5.  Class  Standing: 

A.  Freshman 

B.  Sophomore 

C.  Junior 

D.  Senior 

E.  Other 

6.  GPA:  _ 

7.  How  often  do  you  play  video  games? 

A.  Never 

B.  Have  played  a  couple  of  times 

C.  Several  times  a  year 

D.  Several  times  a  month 

E.  Several  times  a  week 

F.  Everyday 

8.  How  often  have  you  worked  with  personal  computers? 

A.  Never 

B.  Have  played  a  couple  of  times 

C.  Several  times  a  year 

D.  Several  times  a  month 

E.  Several  times  a  week 

F.  Everyday 
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Rate  your  experience  with  personal  computers: 

A.  Little  or  none 

B.  Know  a  little;  have  played  some  computer  games;  know  some  word  processing  and 
some  other  software 

C.  Know  quite  a  bit;  have  played  computer  games,  know  Internet  access,  know  word 
processing  well,  used  other  software  packages 

D.  Expert;  have  played  computer  games;  know  Internet  access,  word  processing,  other 
software  and  some  programming 

How  well  do  you  know  the  people  who  are  your  team  members  in  this  experiment? 

A.  Not  at  all 

B.  Casual  acquaintance 

C.  Friend 

D.  Very  close  friend 

Have  you  ever  served  in  the  military  or  had  experience  with  a  simulated  radar  task? 

A.  No 

B.  Yes 

If  yes,  please  explain:  _ 
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APPENDIX  B 
TRAINING  MATERIALS 
AIRTANDEM  INSTRUCTIONS 

During  the  next  few  hours,  you  will  be  playing  a  radar  simulation  game.  First,  you  will 
be  given  instructions  on  the  task,  and  then  you  will  be  asked  to  play  the  game.  Then,  you  will 
n  watch  a  short  multimedia  presentation  that  should  help  you  to  improve  your  performance.  This 

research  should  give  us  insight  into  how  we  can  use  multimedia  to  enhance  team  training.  So,  I 
^  ask  that  you  please  give  your  full  attention  and  try  to  do  your  best  in  this  experiment. 

First,  you  will  be  trained  on  the  task  of  operating  the  program.  You  will  have  an 
opportunity  to  practice  with  the  task  before  any  record  is  kept  of  your  performance.  Then  you 
will  be  asked  to  play  the  game  for  several  trials.  Once  you  have  learned  how  to  play  the 
simulation,  you  will  watch  a  multimedia  presentation.  After  the  presentation,  you  will  play  two 
more  times  and  complete  a  short  questionnaire.  Please  feel  free  to  ask  questions  anytime 
throughout  training,  because  it  is  very  important  that  you  understand  how  to  perform  this  task. 

Scenario  Script 

During  the  simulation,  you  will  be  monitoring  a  radar  screen  onboard  the  Eagle  Eye,  an 
airborne  early  warning  airplane.  Your  aircraft  is  currently  positioned  in  the  Misty  Gulf,  where 
tensions  are  high.  The  US  Navy  is  about  to  deploy  a  strike  package  and  there  have  been  reports 
of  enemy  contacts  in  the  area.  Your  craft  is  on  alert  and  has  orders  to  prosecute  hostile  targets. 
Your  aircraft  is  manned  by  two  radar  operators,  referred  to  as  Alpha  and  Bravo. 

Alpha  and  Bravo  have  the  same  radar  scope  in  the  middle  of  their  screens.  Your  aircraft 
is  in  the  center  of  the  radar  scope  in  the  middle  of  the  computer  screen.  (POINT  TO  THE 
SHIP).  Surrounding  your  ship  will  be  a  series  of  fuzzy  “blips”  called  targets.  (POINT  TO 
THE  TARGETS).  Your  team’s  task  is  to  assess  information  pertaining  to  the  fuzzy  images  or 
targets  on  your  radar  scope.  More  specifically,  three  decisions  must  be  made  about  each  target. 
The  first  decision  requires  you  to  determine  what  type  of  craft  you  have  hooked.  The  second 
decision  requires  you  to  determine  the  force  of  this  contact.  The  third  decision  requires  you  to 
^  identify  the  craft’s  intent.  This  information  will  allow  you  to  identify  these  targets  and  make 

decisions  about  whether  to  clear  them  or  shoot  them  to  gain  points  for  your  team.  Your  objective 
,  for  this  experiment  is  to  maximize  your  team  score. 

There  is  no  need  for  you  to  actually  memorize  all  of  the  information  and  rules  you  are 
about  to  receive.  You  may  refer  to  Posters  A,  B,  and  C  (POINT  TO  POSTERS)  while  playing 
the  game.  Poster  A  indicates  the  type  of  craft,  which  will  be  air,  land,  or  sea.  Poster  B  indicates 
civilian,  unknown,  or  military  craft.  Poster  C  indicates  intent  to  your  own  ship  with  the  possible 
intent  being  friend,  fleet,  or  foe.  Upon  reference  to  the  computer  screen  you  will  notice  the  drop 
down  menus  A,  B,  and  C,  which  correspond  respectively  with  posters  A,  B,  and  C. 
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An  example  of  how  to  use  the  posters  is  as  follows:  To  determine  Initial  Climb  or  Dive 
Rate  poster  A  is  utilized  (POINT  TO  POSTER  A).  This  is  used  to  determine  whether  a  contact 
is  climbing  or  diving.  Positive  numbers  mean  climbing,  indicating  Air  or  Land  and  0  means 
level,  indicating  Sea.  You  will  need  to  go  through  similar  steps  to  make  accurate  decisions  for 
each  of  the  posters. 

Now,  let’s  look  at  the  menus.  Menu  OP  is  for  set-up  purposes.  (POINT  TO  MENU 

OP). 

f 

Menus  A,  B,  and  C  (POINT  TO  MENUS)  are  your  information  fields.  This  is  the 
information  that  is  being  picked  up  by  your  sensors.  You  will  need  to  access  these  fields  to  carry  -i 

out  your  task.  You  will  use  the  menus  to  get  the  information  you  will  need  to  determine  the  type 
of  craft. 

Menu  Description 

Next,  I  am  going  to  walk  you  through  an  example. 

To  begin,  we  first  need  to  start  the  system.  To  do  that,  you  will  use  the  right  mouse 
button  to  click  on  the  OP  Menu.  As  you  can  see,  this  reveals  four  choices:  Start  Exercise, 

Shutdown  System,  Zoom  In,  and  Zoom  Out.  Please  do  not  click  on  Shutdown/End  System.  If 
you  mistakenly  select  this  option,  an  information  box  will  appear  asking  if  you  want  to  quit  the 
simulation.  At  that  point,  type  “n”  for  no  and  the  box  will  disappear.  Zoom  In  and  Zoom  Out 
will  be  discussed  later.  Go  ahead  and  click  on  Start  System.  Please  be  careful  when  using  the 
Zoom  In  and  Zoom  Out  options  so  as  to  avoid  accidentally  selecting  the  shutdown  system 
option.  You  can  click  on  OP  again  to  close  the  menu. 

Now  let’s  continue.  To  select  a  target,  use  the  mouse  to  position  the  pointer  directly  over 
the  fuzzy  image,  then  click  on  it  using  the  left  mouse  button.  You  will  know  if  you  hooked  the 
target  because  it  will  change  from  white  to  green.  The  number  of  the  target  which  you  have 
hooked  appears  in  the  lower  right  hand  comer  of  your  radar  screen.  You  have  hooked  target  #  2. 

If  you  change  your  mind  and  wish  to  hook  a  different  target  simply  position  the  mouse  pointer 
over  that  target  and  click  on  it  using  the  left  mouse  button.  Now,  please  hook  the  target  #  1  and 
we  will  work  on  this  one  as  a  team. 

A 

Now  click  on  Menu  A.  We  will  take  a  look  at  the  five  pieces  of  information  that  you  will 
need  to  determine  the  type  of  craft  ( air,  land,  sea)  for  each  target.  They  are  current  speed,  initial  ^ 

altitude/depth,  initial  climb/dive,  communication  time,  and  signal  strength.  Click  on  Current 
Speed  using  the  right  mouse  button  and  continue  to  press  the  mouse  button.  The  current  speed  of 
this  craft  will  appear  in  the  menu  box.  As  you  can  see,  the  current  speed  for  this  craft  is  1 1 
knots.  According  to  poster  A,  this  craft  is  a  sea  craft.  Each  information  field  is  queried  using  the 
same  procedure,  that  is  by  clicking  on  the  menu  bar  with  the  right  mouse  button. 

The  Initial  Altitude/depth  for  this  craft  is  5524  ft  which  indicates  that  is  an  air  craft.  Its 
initial  climb/dive  rate  is  939  ft,  also  indicating  an  air  craft.  The  Communication  Time  is  1 16  sec 
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indicating  sea,  while  the  Signal  Strength  is  medium  designating  an  air  craft.  Since  we  have  three 
pieces  of  information  which  suggest  that  the  craft  is  an  air  craft  and  only  two  suggesting  it  is  a 
sea  craft,  we  will  decide  that  this  target  is  an  air  craft.  After  querying  all  this  information  and 
making  your  decision,  you  must  select  the  “change  symbol  type”  option  from  the  menu  and  label 
the  target  as  either  a  air,  land,  or  surface.  In  this  case,  we  will  label  it  an  air  craft. 

Now  click  on  Menu  B  in  order  to  determine  the  force  of  this  target.  We  will  take  a  look 
at  the  five  pieces  of  information  that  you  will  need  to  determine  whether  the  targeted  craft  is 
civilian  or  military.  These  are  Initial  Bearing,  Initial  Range,  Intelligence,  Direction  of  Origin, 
and  Maneuvering  Pattern.  The  same  procedure  is  applied  when  determining  if  a  craft  is  civilian 
^  or  military.  That  is  Click  on  Initial  Bearing  using  the  right  mouse  button.  A  menu  box  will 

appear  indicating  that  the  craft’s  initial  bearing  is  1 17.  According  to  Poster  B,  this  indicates  a 
civilian  craft.  The  Initial  Range  is  2.5  nautical  miles  which  indicates  a  civilian.  Intelligence  is 
platform  which  designates  military.  The  Direction  of  Origin  is  Red  Sea,  which  indicates  that  it  is 
military.  The  Maneuvering  Pattern  is  Code  Delta,  which  also  indicates  military.  Since  three 
pieces  of  information  suggest  a  military  craft  and  two  indicate  a  civilian  craft,  we  will  decide  that 
the  aircraft  is  military.  Then  choose  the  “change  military  status”  option  and  label  the  target  as 
civilian  or  military.  In  this  case,  we  will  label  it  military. 

Finally,  we  will  determine  the  crafts’  intent.  Now  click  on  Menu  C.  We  will  look  at  five 
pieces  of  information  that  you  will  need  to  determine  the  intent  to  your  own  craft.  These  include 
countermeasures,  electronic  warfare,  threat  level,  response,  and  missile  lock.  Countermeasures 
for  this  target  are  IRST,  according  to  poster  C,  this  indicates  a  the  target  is  part  of  the  Fleet. 
Electronic  warfare  is  big  bulge  radar  which  also  indicates  Fleet.  Threat  level,  however,  is  1 
which  suggests  a  friend.  Response  is  given,  designating  friend.  Finally,  missile  lock  is 
unengaged  meaning  Fleet.  Since  three  pieces  of  information  indicate  Fleet,  we  will  decide  that 
this  military  air  craft  is  part  of  the  Fleet.  Now,  you  must  choose  the  “change  intent”  option  and 
label  the  target  as  Fleet. 

The  final  action  that  must  be  taken  to  prosecute  this  target  is  the  final  engagement.  The 
rules  for  determining  the  final  engagement  action  are:  A  friend  does  not  pose  a  threat  so,  they  are 
cleared.  A  foe  means  harm  to  your  ship  and  you  should  choose  “shoof  ’  it.  If  you  decide  that  the 
craft  is  Fleet,  it  is  part  of  the  Navy’s  strike  package-  do  not  shoot  it  down,  choose  “clear”  from 
the  menu. 

4 

Now  let’s  select  Final  Engagement  from  Menu  C  and  clear  this  craft.  When  you  click  on 
V*  clear  continue  to  hold  down  the  right  mouse  button  to  determine  if  you  correctly  engaged  the 

target.  Are  there  any  questions? 

Now,  let’s  return  to  the  issue  of  zooming  in  and  zooming  out.  Click  on  the  OP  menu,  and 
you  will  use  zoom  in  to  view  a  smaller  range  of  the  total  radar  screen.  You  should  notice  that 
range  of  your  radar  screen  changes  from  32  nautical  miles  to  16  nautical  miles.  You  will  use 
zoom  out  to  view  a  larger  range  of  the  total  radar  screen.  You  may  want  to  zoom  out  to  get  a 
better  view  of  possible  incoming  targets.  Now,  zoom  out  to  64  nautical  miles.  Do  you  see  the 
other  target? 
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At  this  time,  I  would  just  like  to  take  a  few  moments  to  provide  you  with  a  little  more 
information  about  the  decision  process.  You  are  likely  to  get  conflicting  information  on  most 
targets.  As  we  did  in  our  example.  To  make  an  accurate  decision,  you  will  need  at  least  three 
pieces  of  information  indicating  the  same  type  of  craft.  For  example,  if  you  found  that  two 
information  pieces  indicated  Sub,  and  three  indicated  Air,  for  a  particular  target,  then  you  could 
confidently  judge  the  target  as  an  aircraft.  Keep  in  mind  that  each  of  the  five  types  of 
information  are  of  equal  importance  for  you  to  make  an  accurate  decision.  In  other  words,  you 
should  not  weigh  one  type  of  information  more  heavily  than  another. 

r 

This  simulation  has  two  additional  features  you  should  know  about;  the  clock  and  the 
score  box.  As  you  can  see,  there  is  a  clock  on  the  upper  left  hand  comer  of  the  computer  screen.  .j 

(POINT  TO  CLOCK).  You  will  be  given  twenty  minutes  for  each  trial.  On  the  upper  right 
hand  comer,  you  will  see  a  box  containing  your  score  (POINT  TO  SCORE  BOX).  The  box 
labeled  T  score  is  the  teams  score.  If  the  team  is  correct  in  all  three  decisions,  then  clearing  or 
shooting  the  target  will  earn  the  team  100  points.  If  any  one  of  these  decisions  is  incorrect,  the 
team  will  lose  100  points.  The  two  of  you  as  a  team  will  accumulate  one  score  throughout  task 
performance. 

We  have  taken  you  through  each  step  in  identifying  and  engaging  targets.  If  you  have  any 
questions,  do  not  hesitate  to  ask  them  at  this  point  because  it  is  very  important  that  you 
understand  how  to  perform  this  task. 
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APPENDIX  C 

TRAINING  INTERVENTION  MATERIALS 


Slide  1 


Player  1 :  Let’s  take  a  moment  to 

look  at  all  the  targets  and 
determine  which  ones  could  be 
threats  to  us. 

Player  2:  I  think  that  is  the  best 
strategy.  Let’s  take  a  moment  to 

look  at  all  the  targets,  determine 
if  any  are  flying  in  formation,  or 
if  any  are  heading  straight  toward 
us. 

Player  1 :  Let’s  also  zoom  out  and 
see  how  many  other  targets  could 
be  potential  threats... and  there  seem 
to  be  a  lot.  So,  let’s  zoom  in  and 
concentrate  on  those  closest  to  us. 


Slide  2 


Player  2:  Have  you  identified  any 
problems? 

Player  1;  There  is  a  target  heading 
straight  toward  us  and  he’s 
coming  at  a  quick  speed. 

Player  2:  I  see  him. 

Player  1:  I  hooked  him.  He’s  target 
28.  Let’s  query  his  information. 
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Slide  3 
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Player  2: 1  have  identified  a 
problem?  There  seem  to  be  two 
craft  in  formation  and  heading 
toward  us. 

Player  I :  I  see  him.  Let’s  hook  the 
guy  closest  to  us.  It  is  track  15. 


Slide  4 


Player  1 :  He  was  one  of  our  guys, 
but  let’s  check  the  guy  behind  him 
to  make  sure  he’s  also  not  a  threat. 

Player  2:  Ok.  I  have  hooked  him. 
He’s  number  7. 

Player  2:  Let’s  query  that 
information. 
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Slide  5 


1 

I 


Slide  6 
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Slide  7 


Player  1: 1  just  identified  another 
problem.  There  was  a  target  that 
popped  up  close  to  the  perimeter. 

Player  2:  I  saw  him.  Let  me  go 
ahead  and  hook  him.  He’s  track  34. 


r 


Slide  8 


Player  2:  Have  you  identified  any 
other  problems? 

Player  1 :  I  think  I  see  two  craft  in 
formation  toward  the  outside, 
around  290  degrees. 

Player  2:  I  see  him.  I  hooked  him. 
He’s  number  29. 
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Slide  9 


I 


Player  1 : 1  think  we  should  also 
check  the  target  that  was  behind 
him.  He  may  also  be  a  threat. 

Player  2:  I  hooked  him.  He  is 
number  32. 


Slide  10 
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V 


Player  2:  Have  you  noticed  if  any 
new  targets  have  popped  up? 

Player  1 :  One  just  popped  up  at 
around  210  degrees. 


Player  2:  I  see  him.  I  have  hooked 
him.  He  is  number  6. 
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Slide  12 
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Slide  13 


Player  1 :  There  is  a  target  coming 

in  quickly  toward  us,  that  looks 
like  a  potential  threat  I  think  we 
should  query  that  information. 


W' 


Player  2:  I  see  him  at  the  bottom  of 
the  screen.  I  have  hooked  him.  He 
is  number  1. 

Player  1 :  Ok,  I  have  him  hooked 


Slide  14 
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Player  1:  This  situation  is  evolving 
nicely.  I  think  we  should  maybe  zoom 
out  and  see  what  else  is  coming 
toward  us  or  is  active. 

Player  2:  Ok,  I  am  zooming  out  now. 

Player  1 :  Actually,  I  see  two  targets 
which  look  somewhat  suspicious. 

They  are  at  approximately  100  degrees. 
They  are  both  traveling  together. 

Player  2:  Do  you  think  they  pose  a 
threat  to  us? 

Player  1 : 1  think  we  should  check  it 
out. 

Player  2:  Ok,  I  have  hooked  number 

M _ 
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Slide  15 


Player  1 :  Let’s  go  ahead  and  check 
the  craft  that  was  with  him,  just  to 
be  on  the  safe  side. 

Player  2:  I  hooked  him  also.  He’s 
number  27. 
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APPENDIX  D 

TRAINING  INTERVENTION  QUESTIONNAIRE 


Identification  No. 


The  following  questionnaire  is  designed  to  inform  us  about  the  effectiveness  of  our 
multimedia  training  intervention.  Please  circle  the  number  which  corresponds  to  your  perception 
of  the  relative  helpfulness  of  the  multimedia  presentation.  Please  refer  to  the  following  scale  for 
the  assignment  of  numerical  ratings. 


1  2  3  4  5 

Not  at  all  Slightly  Moderately  Very  Extremely 

1.  How  helpful  was  the  training  intervention  in  teaching  you  1  2  3  4  5 

which  targets  to  select  for  prosecution? 

2.  How  helpful  was  watching  another  team  playing  this  1  2  3  4  5 

simulation? 


3.  Do  you  feel  that  watching  another  team  perform  the  1  2  3  4  5 

simulation  helped  you  to  coordinate  better  with  your 
teammate? 


4.  After  watching  the  presentation,  would  you  agree  that  you  1  2  3  4  5 

adopted  the  strategy  used  by  the  team  you  watched? 

5.  Did  your  presentation  contain  text  that  was  colored  or  Yes  No 

highlighted? 


6.  What  color  was  the  highlighted  text? 

A.  Blue 

B.  Green 

C.  Yellow 

D.  Red 

E.  Purple 
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7.  Please  circle  those  phrases  that  were  highlighted? 

A.  noticed  that  there  is  a  craft  just  hovering 

B.  two  craft  in  formation 

C.  I  have  hooked  the  target 

D.  That’s  the  best  strategy 

E.  Look  at  all  the  targets  and  determine  which  could  be 

threats 

F.  Check  the  guy  behind  him 

G.  Find  out  if  he  is  a  threat 


H.  popped  up 

I.  identified  a  problem 

J.  look  suspicious 

K.  Let’s  zoom  out 

L.  Let’s  query  him 

M.  He’s  track  34 


8.  Did  your  presentation  contain  colored  or  highlighted  targets?  Yes  No 

9.  What  color  were  the  highlighted  targets? 

A.  Blue 

B.  Green 

C.  Yellow 

D.  Red 

E.  Purple 

10.  Do  you  typically  highlight  passages  in  your  textbooks?  Yes  No 

1 1 .  How  helpful  do  you  feel  the  highlighting  was  in  making  1  2  3  4  5 

behaviors  more  obvious  to  you? 

12.  Overall,  how  helpful  do  you  think  the  training  intervention  1  2  3  4  5 

was? 

13.  How  strongly  would  you  recommend  using  this  technique  1  2  3  4  5 

for  team  training? 


( 

J 


h 
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